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A formula is derived for the capacity of a multi-input, multi-output 
linear channel with memory, and with additive Gaussian noise. The 
formula is justified by a coding theorem and converse. The channel 
model under consideration can represent multipair telephone cable in- 
cluding the effect of far-end crosstalk. For such cable under large signal-to- 
noise conditions, we show that channel capacity and cable length are 
linearly related; for small signal-to-noise ratio, capacity and length are 
logarithmically related. Crosstalk tends to reduce the dependence of 
capacity on cable length. Moreover, for any channel to which our capacity 
formula applies, and for large signal-to-noise ratio, there is an asymp- 
totic linear relation between capacity and signal-to-noise ratio with 
slope independent of the channel transfer function. For small signal-to- 
noise ratio, capacity and signal-to-noise ratio are logarithmically related. 
Also provided is a numerical evaluation of the channel capacity formula, 
using measured parameters obtained from an experimental cable. 

I. INTRODUCTION AND STATEMENT OF RESULTS 

Our problem is to calculate the capacity of a multi-input, multi- 
output linear channel with additive Gaussian noise, and to justify the 
formula by a coding theorem and converse. Specifically, we consider 
the following channel. The channel input and output are sequences 
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\x(?i) } _ «, { y(n) } - » of real s-vectors* related by 

y(n) = £ fe(n - fc)x(/c) + z(n), (1) 



fc=-» 



where {h(m) }£=-*, is a fixed sequence of real s X s matrices (the 
indicated operations being ordinary matrix arithmetic), and {z(n)}!„ 
is a sequence of Gaussian random s-vectors for which Ez{n) = 0, and 

Ez{ri)\_z(n - m)~\ t = r(m), — °o < n, m < *>, (2) 

where r(m) is an s X s matrix. The motivation for this problem is that 
it is a model for a multipair telephone cable. 

The first sections of this paper are highly theoretical; the formula 
for channel capacity is carefully and precisely established by means of 
several rather technical theorems. In the final section, Section IV, we 
discuss some engineering implications of our formula in terms of its 
asymptotic behavior, and evaluate the capacity numerically with 
measured parameters obtained from an experimental multipair 
telephone cable. 

A code for this channel with parameters (M, N, S, X) is a set of M 
pairs {(x,-, B x ))?=u where x t - =(•••, x\{-2), x l t (-l), xKO), aftl), • • •)' 
is a sequence of s-vectors that satisfy the following: 

xt(n) = 0, for n < 0, and n ^ N, (3a) 

i N £ \\xi(n)\\^ S, (3b) 

iV n =0 

(where || • || denotes Euclidean norm) and the B { are (measurable) 
subsets of (R' N with the following property. Let y,- = (• • •, y\{ — 1), 
yKO), 3/|(l), • • •)' be the channel output vector which results when the 
channel input is x,-, i.e., 

00 

y%(n) = £ h(n — k)xt(k) + z(n) 

k— oo 

= ZHn -k)xi(k) +z(n). 

fc=0 

Lety t w = ft/KO), • • •, y\{N - 1)]' G W N . Then 5,(1 ^ i ^ M) must 
satisfy 

P ei = Prfyr G B t } ^ X. (4) 



* Vectors will be taken to be column matrices unless otherwise indicated. 
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Thus the x, are code words and JS^is the set of output iV-vectors 
which are decoded at the channel output as x,. Inequality (4) expresses 
the requirement that the error probability, given that x, is transmitted, 
does not exceed X. 

A number p ^ is said to be "^-admissible" (S ^ 0) if for all 
X > there is an N such that there exists a code with parameters 
([2 Arp 3> N} S, X). The channel capacity Cs is defined as the supremum 
of <S-admissible rates. Our problem is the calculation of Cs, which we 
shall solve provided the channel satisfies the following conditions: 

(i) The filter [h(m)\: We assume that the filter is causal, i.e., 
h(m) = 0, m < 0. We also assume that 

t KAMI < «, (5a) 

m— 

and that there exists a B > such that for m > 0, 

\\h{m)\\ ^ Bmr\ (5b) 

where the Euclidean norm "|| • ||" of a matrix is the square root 
of the sum of the squares of its entries. From (5), the (discrete) 
transfer function, 

H(d)= £ h(n)e ine , -ir ^ 6 S ir, (6) 

n = 

exists and is continuous. H(6) is an s X s matrix. We assume 
that for -it ^ 6 ^ 7T, det H(d) ^ 0. 
(ii) The noise covariance: We assume that the covariance sequence 
r(-) satisfies 

t ||r(n)|| < oo, (7) 

n— oo 

so that the (discrete) power spectral density 

B($) = £ r(n)e inB , -w ^ d ^ w (8) 

n— oo 

exists and is continuous. R(6) is an s X s matrix. We also 
assume that for — ir ^ ^ it, det R(8) j* 0. 

We can now give the capacity formula. Let the s X s matrix t 
T(0) = H(d)- i R(6)H(d)-* ) and let Xi(0), X 2 (0), • • •, X.(0) be the eigen- 



t For any nonsingular complex matrix A, A~* is the transpose conjugate of A -1 . 

GAUSSIAN CHANNEL WITH MEMORY 747 



values of T(8), -* ^ d £ ir. Then X,(0) > 0,1 < j ^ s, -ir ^ d ^ t. 
Let S ^ be given, and let K s be the (unique) positive number such 
that 

'" dOmax[p, K s - X,(0)] = 5. (9a) 



Then 



1 s /"' 

^'s,?,/.'^" 1 -^ 108 ^)- (9b) 

Our main result is the following. Consider the channel defined by 
(1) with H = Hi(d) and R = Bi(6). Then Cs is calculated from (9) 
with T(d) = H 1 (6)- l R 1 (d)H 1 (d)-*. 

Theorem la (Converse): Let p ^ be an S-admissible rate for this 
channel. Then p ^ Cs. 

Theorem lb (Direct-Half): Let S ^ 0, e > and p(0 ^ p < Cs) be 
arbitrary. Then for N sufficiently large, there exists a code with parame- 
ters (M, N, S, X) where 

M ^ e" N and X ^ e. 

Sections II and III of this paper are concerned with the proof of 
Theorem 1 . Section IV is concerned with the asymptotic behavior and 
numerical evaluation of the channel capacity formula (9), with specific 
attention to multipair telephone cable. 

Theorem 1 is very similar to the results on continuous-time Gaussian 
channels due to Holsinger and Gallager. 1 In fact, for the special case 
in which H(d) is the s X s identity, the theorem follows immediately 
from the analysis in Ref. 1. We suspect that it might be possible to 
obtain all of Theorem 1 by paralleling Gallager's techniques for this 
discrete case, although such an approach is somewhat more cumbersome 
than the approach followed here. Furthermore, the present approach 
lends itself immediately to broadening the model to consider the effects 
of intersymbol interference from previous channel uses, as Gallager's 
approach does not. 2 In fact, to establish Theorem lb for the inter- 
symbol interference channel we require only to add in one of our 
lemmas a term "y 3 " (representing the effect of previous channel 
uses), and to show that its norm | Hirslll = o(A ri ). 

Careful analysis of the proof of Theorem 1 will indicate that the 
conditions on the filter \h(m)} given in Section I can be replaced by 
simply requiring that the filter have a causal inverse {g(m)\, such that 
g(m) = 0, m ^ m (i.e., finite memory). Thus, our results contain a 
generalization of those given by Toms and Berger. 3 
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II. NOTATION AND MATHEMATICAL PRELIMINARIES 

Let l^(a, b), s = 1, 2, • • •, — ■» ^ a < b ^ <», where a, b are 
integers, be the set of sequences \x(n) } b n=a where x(n) is a real s-vector. 
Such sequences will often be written as column matrices x = [z'(a), 
x l (a + 1), ••',x'(b)2 t . Let ||-|| denote ordinary Euclidean norm in 
s-space, and f or s X s matrices A, let ||A|| be the Euclidean norm (i.e., 
the square root of the sum of the squares of the components of A). 
For sequences x £ fiP(o, b), the (Euclidean) norm is 

IIMII- [jElMiOI']*- do) 

The space W{a, b) is a Hilbert space with the obvious inner product, 
written (x, y) = IJ =a x'(n)y(n), x,y £ W(a, b). For x £ W(- », °°), 
denote by x (JV) £ Ji 5) (0> N — 1) the column matrix 

x<"> = [>'(0), &<(1), • • •, x'(iV - 1)]'. (11) 

We denote operators on UP (a, b) by script letters, e.g., fF. We define 
the norm of JF by 

Iff I =. « 1Y) 11k x H1 / 12 \ 

If | JF | < oo } we say that 'S is bounded. An operator JF is said to be of 
a convolution type if, for x £ W(a, b), 

(ffx)(n) = £ /(n - k)x{k), n = a, ■ ■ ■, b, (13) 

where {/(n)}*-S is a fixed sequence of s X s matrices and the indicated 
operations are ordinary matrix arithmetic. Let L be the set of convolu- 
tion-type operators on ljp(— °° , °° ) for which 

f ||/(n)|| < ». (14) 

71 =—00 

For operators JF in L the transfer matrix 

F(0) = £ f(n)e inB , -w^d^T (15) 

is well defined. F(0) is an s X s matrix and is continuous (in Euclidean 
norm) for — tt ^ 6 ^ n. Concatenation of operators JFi, IF 2 £ L, 
defined by sequences { /i(n) } and { /2(n) } , results in a convolution-type 
operator 5F 3 = OVJFi in L defined by the sequence {/3(w)} where 
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f3(n) = L*°=-« Mk)M n ~ k )- Further, the corresponding transfer 
functions satisfy F 3 (d) = F 2 (d)F 1 (e). Other relevant properties of the 
class L are given in the theorem below, the assertions of which are 
generalizations of well-known scalar results. The proof is given in the 
appendix (Section A.l). 

Theorem 2: Given JF £ L, defined by {/(»)} or F{6), then 

a. |ff| ^ £«=-• ||/(n)|| < », so that EF is bounded on Iff 1 (— », °°). 

b. \ff\ ^ max \\F(0)\\. 

c. // fF z's self -adjoint, then for allx £ Z^(— °o, oo), 

E x'(n)/(n - m)z(m)| =S ( max ||F(0)||)|||x||| 2 . 

d. SF /ias a bounded inverse denoted tf -1 # and onZy if det F(d) 7* 0, 
— ir^d^T, in which case the transfer matrix corresponding to 
JF" 1 U lF(0)Jr 1 , and fF" 1 £ L. 

Let z = [• • •, 2'(-l), z'(0), 3'(1), ■•-,]' be a sequence of random 
s-vectors with covariance 

Ez(n)z(n — m) 1 = r(m), 
where r(m) is an s X s matrix. Under the assumption that 

£ ||r(w)|| < oo, (16) 

the power spectrum 

R(6) = £ r{m)e ime , -v ^ d ^ v, (17) 

m=— m 

is well denned. Let £F be an operator in L corresponding to the transfer 
matrix F(6). Thenz = JFz is a sequence of random s-vectors with covari- 
ance { r(m) } and corresponding power spectrum A(d) = F(d) R(d)F(6) *. 
Let z be a sequence of zero mean Gaussian n-vectors with covariance 
r(m) satisfying (16) and power spectrum R(d). Let \ m ia(0) be the mini- 
mum eigenvalue of the matrix R(9). Let i<*> = [>'(0), «'(!), ■••, 
z l (N — l)]'be a segment of z of duration N. Then there exists an 
(N-s) X (N • s) matrix TV such that 

w = Tnz (n) , (18) 

is "white," i.e., £ww' = /#.,. The indicated operation in (18) is 
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matrix multiplication. The only property of T N which we need here 
is that for any N • s-vector u 

W f -W g [ min Lffl ] >II|U|11 ' (19) 

Finally, let 3C be the convolution-type operator on IP ( — °° , °° ) 
defined by {h{m)\ (Section I). Note that by (5) 3C E L, so that by the 
assumption following (6) and by Theorem 2d it has an inverse, say 
S = JC" 1 , of the convolution type in L. Let {g(n)}- M be the sequence 
which defines g. Let the operator Qn(N = 0, 1, 2, • • •) be defined by 

(9*x)(n)= £ ff(« - *)*(*)i - « < n < oo. (20) 

We conclude this section by stating as lemmas two known results. 
We explain in the appendix (Section A. 2) how to obtain these results 
from published material. 

Lemma 3: For the special case when H(8) = /„ {the s X s identity) and 
R(B) = R 2 (9) [ so that T = H~ l RH-* = Rtffl}* sa V tfiat x •'■ a random 
channel input sequence for which x(n) = 0, n < 0, n ^ N, and i£|||x||| 2 
^ NS(S > 0, N = 0, 1, 2, • • •)• Let y be the corresponding channel 
output sequence. Then, the mutual information 

/{x,y} ^NC S 

(where Cs is calculated with T(6) = #2(0)). 

Lemma 4-' For the special case where H(9) = I, and R(6) = Ri{6), then 
we have a stronger version of Theorem lb: Let S ^ and p (0 ^ p < Cs) 
be arbitrary, where Cs is calculated with T(d) = Rz(6). Then, for N=l, 
2, • ■ • , there exists a code with parameters (M, N, S, X) where 

M ^ 2>- v and X ^ Ae~ BN , A, B > 0. 

III. PROOF OF THEOREM 1 

3.1 Converse 

Let [(xt,Bi)]jf be a code with parameters (M, N, S, X) for the 
channel of (1) with H = Hi(6) and R = Ri(6). Let x be the random 
sequence which results when x = x,- with probability 1/M (1 ^ i ^ M). 
Let y = 3Cx + z be the corresponding output sequence, and y (JV) 
= (2/(0)', • • -,y(N — 1)')'. The theorem will follow in the standard way 
from the Fano inequality (see, for example, Ref. 1) if we can show that 

/{x,y<< v >} ^NC S . (21) 
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But 

I{x, yWJ ^ /{x, y} = I{x, Xr*y} = I{x, x + z}, (22) 

where z = 3C _1 z is a stationary Gaussian random process with power 
spectrum r(0) = Hr 1 (e)Ri(d)H l -*(6). Thus, we can apply Lemma 3 
(since x satisfies the required hypotheses) with R 2 (0) = T(6) to ob- 
tain (21). Hence, the theorem follows. 

3.2 Direct-half 

Consider again the special case of our channel where H (0) = I, and 
R(6) = R 2 (6). The idea behind the proof is to construct codes for the 
general case (H, R arbitrary) by modifying codes (whose existence is 
guaranteed by Lemma 4) which are known to be good for this special 
case. We proceed as follows. Let { (x„ £<)}f be a code for this channel 
with parameters N = Ni and S = Si. Then, for 1 ^ i ^ M, we have 

where the superscript operation is defined by (11). Let T N be the 
whitening filter (discussed in Section II) for which T N z KN) = w and 
£ww' = I N .. Letting v, = T N yl m and u, = 2Vx< (iV) , we have 

v, = u, + w. (23) 

Let us assume that the {#,} correspond to the minimum distance 
decoder, i.e., y (JV) G #. if |||v- u,||| < ||| v - Uy||| for all j ^ i, 
where v = 7Vy (JV) . Then 

P ei = Prljl" € Bi\ = PrU Nil* - u,||||£ |||v,- - uy|||} 

Mi 

= Pr U {|||w||| > |||w - (u ; - - uOIHI 
Mi 

= Pr\J |<W, Uy - u t ) ^ J|||Uy - u,-||| 2 }. (24) 

Mi 

Thus, in particular, for all j 9^ i, 

P ei ^ Pr{ <W,Uy-U,-)^^|||Uy-U,||| 2 } 

= •.(Mb - U.IH) = *m\T*br - *F)\\\i (25) 

where 

<|> e (u) = _L /"" e -« ,l2 du 
y2ir Ju 

is the complementary error function. 
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Let Hi(d) and Ri(6) be arbitrary, and suppose that we are given a 
code {(x,-, Bi))jf with parameters (M, N, S, Xi) for use on the special 
channel with H = I, and R(6) = R 2 = T(d) = Hr'RiHr*. Assume 
that the B, corresponds to the minimum distance decoder so that P ei 
is given by (24). We now construct a new code {(x*, B*)}?L l with 
parameters N = N 2 = (1 + S)Ni and 8 = S 2 = a 2 £,/(l + 8) for 
use on the general channel with i/i(0), Ri(6) arbitrary. We set 

azi(n), ^ n ^ Ni - 1, 



x*t(n) = 



0, otherwise, 



where a > 1 and 8 > are arbitrary. Note that we have allowed a 
guard-band or dead-space or width 8Ni following the channel input 
signal. The decoding sets B* (1 ^ i ^ M) are described below. 
The channel output is as in (1) 

y = 3Cx + z, 

where x is the channel input, z is the noise, and 3C is the operator corre- 
sponding to Hi(6). Let 9^2 be the operator defined in (20), and let 

y = SnJ= 9at 2 3Cx 4- Stf.Z 

= x + 2 + & + fc, 

where £i = 9at 2 3Cx — x, z = Qz, and £ 2 = 9j\r a z — z. Let us note that 
y is calculable from y (Ar »> = (y'(0), ■■•,y t {N 2 - 1))'. Further, the 
noise z has power spectrum T(6) = Hr 1 (d)Ri(d)Hr*(8). (In fact, if 
Ki — & = 0, the channel would be equivalent to the special case, and 
the direct-half of the coding theorem would follow from Lemma 4. 
Although this is not the case, of course, we will show that £i and £2 are 
sufficiently small so that Lemma 4 can be applied anyway.) The 
decoding sets B* are denned by: y<" 2) £ B* if y<"»> £ B it lgi^ M. 
Letting y* = x* -f- z + £1 -f- &> (1 ^ i ^ M), and letting T Nl be as 
above, we define 

= aU, + W + Yl + Y2j (27) 

where u, and w are exactly as in (23) and yi = T Nl t,l Nl) (i = 1,2). The 
decoder for the derived code is the minimum distance decoder for the 
v*'s. Now, following the same steps as in (24), we have 

K = Pr\j?* € B\\ = Pr U {|||v; - au,-||| ^ ||jvt - oUyHI) 

= Pr U {<w + yi + Yt, «i - u.) ^ I |||tti - u.lll 2 }- (28) 
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Now, according to (9), the channel of Section I [with arbitrary 
Hi(0) and Ri(0)] and the special channel with H{d) = L and R(6) 
= Hiid^RiWHtid)-* = T(0) have the same Cs. Let e, S > and 
P (0 ^ p < Cs) given, and let {(x it B t )}f be a code with parameters 
(M, JVi, Si, Xi) (as guaranteed by Lemma 4) such that 

M ^ 2" Nl and Xi ^ e. 

We will show that with Ni sufficiently large, the derived code has 
parameter X ^ Xi + e. Thus, we will have found a set of codes with 
parameters (M, N 2 , S 2 , X) with 



(1+5)' " = ^ z |l + 5 

and 

X ^ 2e. 

Since C s is continuous in its arguments, and a may be chosen arbitrarily 
close to 1, and 5 arbitrarily close to 0, the direct-half of the coding 
theorem (Theorem lb) will have been established. 

To show that X for the derived code ^ Xi + e, we must show that 
for each i = 1, 2, • • •, 

Pr{y* t i™ <£ B\) ^ Pr\y\ N « $ #<! + «■ (29) 

Inequality (29) will follow directly from the following lemmas, the 
proofs of which are given at the end of this section. 

Lemma 5: Inequality (29) is satisfied if 

Pr JlllYi + Ytlll ^ ^-^minlllu, - u,||j} ^ 1 - e. (30) 

" i J*j 

Lemma 6: For the codes {(x„ B<)}f, as N -* w, 
min |||u,- - u.lH 2 ^ O(iVi). 

Lemma 7: For arbitrary a > 0, 

Pr{\\\vi + y 2 \\\ 2 ^ aNi] -> 1, as jVi->co. 

Now, from Lemmas 6 and 7, condition (30) in Lemma 4 will be 
satisfied for Ni sufficiently large. This establishes Theorem lb. 

Proof of Lemma 6: Let 



S = 



|yi + rilll ^ 9 min lll u « - u ; 



(31) 



By hypothesis, Pr{S] ^ 1 - e. Since fi, and £* correspond to the 
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minimum distance decoder, we have from (28) 
JVtfW $ B*} fg Pr{£ D [y* (Arj> $ B*} } + Pr[£<} 

£ Pr U £ n (w + Yi + T2, u, - u,-) ^ 5 |||uy - u,-||| 2 ) + e. (32) 

Now if S occurs, 

l(n + t%, u, - u,)| ^ ||| Yl + Y2|||-I!|u y - u.-IU 

s( a -i^)lll«,-«,lll«. 

Thus, the event in the right member of (32) satisfies 
S D (<W + Yi + Y2, U, - U.) ^ I \l\Ui - u,-|||») 

C j< W| u, - u.) ^ |M| UjF - u,-||| 2 - ( a -Zi)||| Uy - u,-|| 

C {<W, Uj -U,-) ^ j||| Uy - U,||| 2 }, 

and (32) becomes 

Prtf™ $ #} g Pr U l(w, uj - u.) ^ i||| Ui - u,||| 2 } + e 

= Pr{yP* C 5il + h 

where the last equality follows from (24). This is (29) so that we have 
proved Lemma 5. 

Proof of Lemma 6: For the codes { (x„ B t ) }f , and N\ — * °° , 
so that from (25), 

».(*llto - oilID ^ ^^*"- 

Since, as 77 —> », $,(77) = e -(i'/2) [i+ (i)] j we j lave 

|||uy - u.-HI 2 ^ 85iV x [l +0(1)], 
which implies Lemma 6. 
Proo/ of Lemma 7: First note that by (19) 

Illy. + r.111 

= W»W> + tf*)in s [ ^^ J'mer + tf»iu 

*[i»Enbs] ,Dlltf " lll + llltf " 1111 
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Thus, it will suffice to show first, || 16**111 = o(N\) asiVi->« ( and 
second, for arbitrary a > 0, Pr{ \\\& Nl) \\\ 2 =" aN t } -»0, as JVi -> oo. 

1. Let x be one of the code vectors in { (x*,B*) } . Then ?i = S^OCx-x, 
so that for — oo < n < » , 

€i(n) = Z 9(n- *)(3Cx)(*). (33) 

fc<0 

Now, since x is one of the code vectors {x*}f, x{n) = for 
n(£[0, Ni - 1]. Also since OC is causal, we have (0Cx)(/c) = 0, 
k < 0, and 

fi(n) = t, g(n- fc)(3Cx)(fc) 

fc=-ATj 

= £ flf(n - fc) £ *(* - J)*(J). (34) 

A=jV 2 i-o 



Next, define the sequence ifc by 

ATi-l 

E 

>=0 



0, k < N 2 

Then (34) is 

*i(n) = t, g(n- k)+{k), 

fc=-w 

i.e., Ki = 94i and 

M* ISl -1114111. ( 35 > 

Now |9| ^ £n«-« \\g(n)\\ < °° from Theorem 2d, and 

1114111*= £ U(k)\\ 2 = £ \\ N E h(k - j)x(j)f 

* / Ni-1 \ / Ni-1 „ „ \ 

= E ( E ll*(* - i)ll")( E 11*0)11'), 

where in the final step we have used the following form of the 
Schwarz inequality: if a is a sequence of s X s matrices, and x is 
a sequence of s-vectors, then 

\\Za(n)x(n)\\*£Z\\a(n)\\*£\\x(n)\\*. 

n n n 
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But since £jV ||x 2 0')|| 2 = |||x||| 2 ^ a*SiNi, we have 

HUM* s rfflW, E "z \\h(k - j)|». (36) 

*=jV 2 j=0 

We will now show that, as iVi — » °o , 

£ "f 1 \\h(k - j)\\* = f N £p(k- j)-^0, (37) 

A_AT 2 y„ *=A^j j=0 

where /(/c) = ||A(fc)||. Expressions (35), (36), and (37) together 
imply that 

Hi (Nl) \\ 2 ^ Hlfclll 1 = o(Ni), as ^-♦oo, 

which is what we set out to establish. It remains to establish (37). 
Now, from (5), £o°° f(k) < oo, and /(&) ^ B/k. Setting 



F(fc) = E /»(j), 



we have 

Q = E E / 2 (fc - j) 

*-Ar 2 j=0 



= E E / 2 (*')= E [F(* - Ni + l) - *•(*)] 

fc-ATj i-Jt-ATi+l * = AT 2 

= E *W ^ E F(k). 



Now' 



E F(k) = kF(k) 

tNi+l 



+ E (* - !)/*(* - l). 



But 
fcF(fc) 



6AT. JAT, sNi 



* We have made use of the formula (summation by parts) 



T.v(k)Au(k) = v(k)u(k) 



-Itt(fc- l)Ay(fc), 



where Au(fc) = u(k) — u(fc — 1). Here u(fc) = F(A;) and u(k) = k. 
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and 



Thus, 



£ (Jb - l)/ 2 (fc - 1) * f kf(k). 

SNi+l SATi 



Q * ( l -±l + l) £ kf(k) ^ ( 1 -^ )b i f(k) - 0, 

\ 5 / SNi \ J SNi 

since L °° /(&) < °°- Thus, (37) is established and we have 
finished the first part. 

2. Since for any a > 0, 

tflll&^lll 2 

Primnw 2 ^ ^n 1 \ ^ a Nl ' 

and since N 2 — (1 + 5)JVi, it suffices to show that 

£|||# V2> III 2 = o(iV 2 ), as iV 2 -> co. 
Now \i = Sat,z — z, where z = 3C -1 z. Hence, 

£[$2(n)&(n)] = £ g(n - i)r{i - j)g l (n - j). 

i,j <0 

hi ^atj 

Let /3 denote any fixed s-vector and define a sequence tfc of s- 
vectors by 

\g l (n - i)p, i < and i ^ N*. 

+ {n ~ l) = | 0, 0|i< iV,. 

Then 

P ELh(n)&(n)l0= £ +<(n - i)r(i - j)*(n - j). 

Since r(-) is a covariance, r(/c) = r'( — fc) for all fc. Application of 
Theorem 2c shows that the double summation above is bounded 
by 

max ||J2(4|| £ H(n-i)\\*. 

—t<B<t i=— co 



Since 



menu 2 = e^wM") - z? £ «&(»)&(*)«. 



b-0 f=l 
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where e„ is the s- vector with jih entry 5,, (u, j = 1, ■ • •, s), the 
desired result will follow if we show that 

-^/3' iV E 1 ^ 2 (n)^(n)]/3= o(l) 

A' 2 n=0 

for any 0. Thus, it suffices to show that 

^- N Z £ U(n - OH* = o(l). 

A 2 n=0 , = -» 

But from the definition of tfc, the last expression can be rewritten 
as 

^"e 1 i i\\m\\ 2 +w-v\n 

iV2 n=0 k=n + l 

Since g is in L, Theorem 2a implies ifc is in Z 2 S) ( — oo , <x> ) . In 
particular, as n — > °o 

£ [|l^)|| 2 +||^(-fc)|| 2 ] = o(i) 

and the desired conclusion follows immediately. 

IV. ASYMPTOTIC BEHAVIOR AND NUMERICAL EVALUATION OF THE 
CAPACITY FORMULA 

In this final section, we discuss some implications of the channel 
capacity formula given in (9). As in the prior sections, the channel is 
assumed to be a multi-input, multi- output channel with memory, 
and with additive Gaussian noise. But in contrast to the previous case, 
instead of a discrete time channel, we consider an equivalent continuous 
time bandlimited channel. 

Specifically, the channel inputs or code words (in a T second block 
coding interval) are vector-valued functions x(-) of dimensions s, 
bandlimited in frequency interval [— W, WJ such that the samples of 
x(-) satisfy 



or £** 



X (£v) =0 > f01 ' 2W <0 
We also have the average power constraint 

r \\x(t)w*dt ^ st, 

J — CO 

where || ■ || denotes Euclidean norm. 
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The channel has s inputs and s outputs and has transfer function 
matrix H(f) for fG[—W, W~\. The additive noise is also vector- 
valued and has power spectral density matrix R(f). Let {X,(/){ 
denote the set of eigenvalues of H- l (f)R(f)[H- 1 (f)1*- 

The capacity of this channel is determined as follows. Let S > 
be given and let K be the unique positive number satisfying 



t l W max[0, K-Hf)3df=S. 

,=i J-W 



(38a) 



Then 



c =il;I> ax ( ' loB! x§)) d/ (38b) 

with C in bits per second. 

Formula (38) can be obtained from the analogous formula (9) via 
application of the sampling theorem. The somewhat tedious derivation 
is carried out for the scalar case (s = 1) in the appendix (Section A.3). 

We consider several implications of (38). Specifically, for large 
signal-to-noise ratio, C is linearly related to signal-to-noise ratio; a 
change in C is proportional to a change in signal-to-noise ratio (in dB). 
Furthermore, the constant of proportionality depends only on the 
product sW and is independent of any other characteristic of the 
channel. For small signal-to-noise ratio, C is logarithmically related to 
signal-to-noise ratio; a change in logioC is proportional to a change in 
signal-to-noise ratio (in dB). The constant of proportionality is 0.1 
for any channel. In the case in which the channel represents multi- 
pair telephone cable with small far-end crosstalk, we show that for 
large signal-to-noise ratio, C is linearly related to the length of the 
cable, and for small signal-to-noise ratio, C is logarithmically related 
to length. Furthermore, the effect of the crosstalk is to reduce the 
dependence of C on cable length. Finally, we present a numerical 
evaluation of (38) using realistic parameters obtained from an experi- 
mental cable consisting of two twisted pairs of wire. 

4.1 Dependence of channel capacity on signal-to-noise ratio 

Define a number N as 

No = ±J- W tmCeRU)df - 

Then N„ represents the noise power per hertz, per dimension, and 
sWN represents the total noise power. We define the following 
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normalized quantities: 

Xt(/) = 2Uf)/N 
K* = 2K/N 
P = S/sWN 
P* = 10 (logic P). 

Now P* is a measure of signal-to-noise ratio in dB. By substituting 
the above quantities into (38), and using the fact that each X* is a 
symmetric function, we have 

P = ^Zj o Taax(0,K*-\* t (f))df (39a) 

and 

C = t J" max(o, log 2 ^ W (39b) 

We will determine the asymptotic behavior of C for both very large 
and very small P. For this purpose we define for every number K* 
sets A,-, i = 1, • • -s as 

A,= U-Xif) ^K*;f^0). 

Let 5,- be the measure of A, and define 5 = (1/s) £ 4«. In addition, we 
require the definition of two average channel characteristics, X and 

log X. Let 

SO ,=l J A, 

and 



logX = ^f / log 2 Xi(/)d/. 

so ,=l ./a, 



Note that 5, X and log X are all functions of P. Let X ra i n = min{Xt(/): 
^ f ^W;l £ i g: s}. Recall from Section I that X min > 0. 
Now, from (39), 

WP 



and 



q 

— = log 2 K* - log X. 



These equations combine to yield 

^ = log2 ( tt + + log2 * " I ° gl (40a) 
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We investigate (40a) for large P. Assume all the X*'s are bounded. 
(Actually, boundedness follows from the hypotheses in Section I.) 
From (39a), K* is an increasing function of P. Let P be sufficiently 
large so that X,(/) ^ K* for all / E [0, W] and i - 1, • • -a. Then 
5 = W and (40a) yields 



C_ 

sW 



= log 2 ( f + *) + ( lo 6 2 * ~ lo S X )- < 40b) 

Note that for given X, s, W, and P, C is minimized when all the X*'s 
are equal and constant. Now, for P»X, we have from (40b), 



C«sT7(log 2 P-logX), 
or 



C « 0.3322 sJFP* - sW log X. (41) 

Now (41) represents a line in the C - P* plane with intercept 
-sW logX and slope 0.3322 sW, and the region of validity of (41) is 
Py>\. Note that the slope is independent of the X*'s; the intercept 
and region of validity are determined by the average channel character- 
istics X and logX evaluated over the whole interval [0, W]. 

We now investigate (40a) for small P. Observe that (WP/SX) + 1 
= K*/\, and as P approaches zero, both K* and X approach X mi „. 
Hence, WP/SK -» 0, as P -* 0. Then (40a) is approximately 

C WP - 

-. W -T*- log 2 e + log 2 X - log X, 

SO 0A 

which can be rewritten as 



-^^[iog 2e + ^ (log ^- 1 - ogX) l- (400 

sT^ L K* - X J X 

We show in appendix Section A.4 that for any channel characteristic 
with X m i n > 0, 

lim log,> - io?x = (42) 

p-o K* — X 

Hence, for small P, 

(log 2 e)s^P 

Amln 

or in logarithmic terms, 

p* 
logio C ^ Tq + logio (sW log 2 e) - logio X min . (43) 
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Now (43) represents a line in the log™ C — P* plane with intercept 
log™ (sW \og 2 e) — logio X m in, and slope 1/10. However, the region of 
validity of (43) is difficult to specify because the location of this 
region depends not just on X mi „ but on the shape of the channel charac- 
teristic in a neighborhood of X m i„. 

4.2 Dependence of channel capacity on cable length 

Suppose that the channel characteristic is a function of a length 
parameter I. Let h and U be two values of I and suppose that P* is 
large and the channel capacity vs P* characteristic is in the linear 
region for l x and Z 2 . Then, from (41), 



C(h) - C(h) tt sWllog \(h) - log X(Z 2 )], 
or 

C(/ 2 ) - C(h) « ± j* log 2 ( |Sii_£ y f) (44) 

where in these relations we have explicitly shown the dependence of 
Xt on length. If P* is very small so that (43) is valid, then 

logio C{h) - log 10 C(h) tt logj ^4tT ) ' < 45 > 

Now consider a multipair cable of length I, with s twisted pairs, 
small far-end crosstalk, and additive white noise. We assume that the 
crosstalk voltage on a single pair due to all disturbers is proportional 
to l*f. Assume also that the attenuation on any pair is proportional to 
lp. If the crosstalk is very small, then a reasonable form for \* is 

e bil/i 

Xi(l ' f) = 1 + Cilf ' 

where 6,- and c,- are constants related to attenuation and crosstalk 
coupling. 4 Define the averages b and c as b = £ bi/s and c = 2] c </ s - 
Now X* can be expressed as 

X* = exp[6,//i - ln(l + c,-/ 2 *)], 

and for small crosstalk, we have c,/ 2 / « 1 for all i and all / and / in a 
range of interest. Then we have approximately 

and, from (44), 

AC! r w 

^ « - log 2 (e) E j o (b 4 fi - Ci p)df, 
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which can be evaluated as 

^«-0.96&*Wl-^) (46) 

(for large P). Thus, C and I are linearly related. Note that the effect 
of the crosstalk is to reduce AC/Al. If cW */2b « 1, then C is effectively 
independent of length. It is theoretically possible to have cW*/2b Xi 1 
and yet have cWH « 1 as required by our analysis. These relations 
imply that 2WUb « 1 is a necessary condition that very small cross- 
talk significantly reduce AC/Al. However, we expect that for realistic 
cable parameters, the reduction in AC/Al due to small crosstalk will 
not be significant. 

Now assume that the channel does not pass dc; i.e., the channel 
characteristic is that given above for / E L7o, /i], a band of strictly 
positive frequencies, and is infinite for frequencies outside this band. 
Let b k = min< b t . Then, for small crosstalk, 

\,„i n ttexpl(b k fh - Ckfl), 

and 

log 10 X min « 0.434b*/j(l - ^ y. 

Thus, for small P, we have from (45), 

Alog 10 C ^_ n.o^./j/i _ W? 



Al 



0.434b* 



y»(i-^). (47) 



As in (46), the effect of the crosstalk is to reduce the dependence of 
channel capacity on cable length. 

4.3 Numerical example 

We consider a two-twisted-pair cable with white additive noise. The 
transfer function matrix H(f) is given by 



rr/rN ,T 1 i2irklif~\ 



with I in feet and / in hertz and 

7 = oV^r"/* + ib2irf. 

The off-diagonal terms in the matrix //(/) represent far-end cross- 
talk. This model is an approximate representation of an experimental 
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Fig. 1 — Channel capacity for experimental cable. Capacity C in units of 10 8 
bits per second is plotted as a function of signal-to-noise ratio P* for various values of 
cable length I. 

two-pair cable. Parameters obtained from measurement are 

k = 1.26 X 10- 12 , 
a = 0.23 X 10- 6 , 
b = 1.48 X 10- 9 . 

This model is valid in the range 10 3 ^ I ^ 50 X 10 3 feet, and 10 8 /2 
g / g 10 7 Hz. 

Since the noise is assumed white, the X*'s are the eigenvalues of 
{H*H)~ l and are given by 

- _ ,. _ .* _ exp(2V27a/>) 



Ax — A2 — A — 



1 + (2ir) 2 f 2 kH 
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The capacity equations (39) become 
and 



iy / 'max[0 ) K*-\*(/)]rf/=P 



rfi / K*\ 

C = 2 / max I 0, log 2 -^ Jdf, 



(48a) 



(48b) 



where / = 10 6 /2 and /i = 10 7 . 

Numerical evaluation of (48) for various values of P and I has been 
performed and the results are given in Figs. 1 and 2 and Table I. The 
figures show C vs P* for various values of I. The C axis is linear in 
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Fig. 2 — Channel capacity for experimental cable. Capacity C in bits per second is 
plotted as a function of signal-to-noise ratio P* for various values of cable length I. 
The C axis is logarithmic. 
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OOcpTj<o^J , 03if5COi-<i-HCN'000'*OOOOOa5CNb-P3'-''-'CNcO^H 

^iot»^c^ec^^pt^eo»ot-;peoiocx)^-<>ooqcNppTj<cx3co 

N»oi'H--rtHHrHNNNNMMWM'J''fl''1<iOiOtOCOCON 



■>*(M(N-l<t'tMC3:t^-t^O-*O00O3tNt*Tt<Tj<cO>-HtX)00'-icpC0O 

^ffloqq(N'qNOMt<;qTi;NrHcqqioqiqHcpcNc;Oi-;N 

^f^^'c^C^C^CSCCMCC^^^iOiOCDCOl^t^CXJodoJci'-H'-H'-H 



I, l-Nt-I-l-Ni^I-t^NNNNXOCOCXMCOOOOOOCXXCC 

WWWWHHWKHHaHHWWHHHHHHHHHHa 
x-nXNci't-iNNfinciMO'i'HoaJCjiNxaoirtN'C 
oqcNU5oinob«oocow>Ort«)ioqHNNM'i;in!ON»qH 

tNMMeC^^^^'cOtO^06c»ai^'^^^H'-H'H'--<^''--''-<C > icN 



t^t-t^t^t^C»OOCX)CX)COCOOOC»CX)CX30000000000000000COCOCO 
CM'0"t l COOOOC033QtN'9 , COOi-<'rc003^HTj(c005'-'^pt^050l 

u^cNocc^o^tN^»ocot^oqp^He^eo'qpt>;00pi--;cNCQiq 

tONX^CJHHrtHHrtrtiHCqNNNNNCNNririMricO 



O0QO00QO0OCOO0O0COCOO0O0O0O0O0Q0C»O0O0C»O00000O00OO0 

WWWWHHMt)9HyaHHHHHWWHHBWWWW 

cOC»-^-*COa!^-t-*c003'--<TjHC002CN-^t^c35CN'tt--a3CN'*S-0 

t^oop^c^coiqp^i»p'-^e<«coiqpt>;c»p'--;cNeoiqpt^;c33 
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Fig. 1, and is logarithmic in Fig. 2. The linear regions discussed above 
are evident in these figures. The asymptotic estimates for large P 
given in (41) and (46) are AC/AP* « 6.3 X 10 6 (b/s/dB), and 
AC/Al « - 70 X 10 3 (b/s/ft). For constant C, AP*/Al & 11 X 10~ 3 
(dB/ft) or about 58 dB/mile. This is the amount of increase of signal- 
to-noise ratio necessary to maintain a fixed level of C as length is 
increased. The asymptotic estimates for small P given (43) and (47) 
are A logi C/AP* « 1/10, and A logi„C/AZ « - 0.353 X 10~ 3 . For 
constant C, AP*/Al W 3.53 X 10" 3 (dB/ft), or about 19 dB/mile. 
These asymptotic estimates are borne out in the numerical evaluation. 
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APPENDIX 

A1 Proof of Theorem 2 

a. Let y(n) = (5x)(n) = Zk f(n - k)x(k), and let £» ||/(n) 
= C < oo . From the triangle and Schwarz inequalities, 

||2/(n)|| 2 ^ (L ||/(n - k)\\-Hk)\\) 2 ^C£\\f(n - fc)||-||s(fc)||'. 

k * 

Hence, 

I 2 = E ||tf(n)||« ^ C£ £ ||/(n - ft)H|s(fc)||' ^ C 2 |||x||| 2 , 



and | £F | £ C. 

b. From Parseval's theorem, 



lll«rlll> / ^(W«lW 
ll|x|11 J ||X(0)|| 8 <W ' 

c. If $ is self-adjoint, then 

|(x. ffx)| ^ iJFl-IIWH 2 ^ max ||F(i)||-|||x|||». 

d. The essence of this result is a matricized version of Wiener's 
well-known theorem on the reciprocal of an absolutely convergent 
Fourier series (see Ref. 5, p. 430). 
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Suppose det F{6) 5^ 0. We show that 5 has a bounded inverse in L. 
Now det F(6) is a scalar function consisting of a sum of products of 
functions each possessing an absolutely convergent Fourier series.* 
Hence, det F(d) and, by Wiener's theorem, [det F($) J" 1 have abso- 
lutely convergent Fourier series. Each element of F~ l (d) is the ratio 
of a minor determinant to det F(6) so each element has an absolutely 
convergent Fourier series. Hence, F~ l {6) has a Fourier series £„ g(n)e inB 
with £n ||flr(n)|| < co. Consequently, $ has a bounded inverse ff- 1 
in L. 

Conversely, let SF have a bounded inverse 5~ l . Then there exists 
an a > such that for all x in ljp (— <x> , oo), 1 1 J fFx[ 1 1 ^ a|||x|||. 
Let X(6) = £„x(n)e ,n9 . From Parseval's theorem, 

f w \\F(e)X(e)\\*de 

< a 2 ^ inf — , 



\ 2 dd 



which implies, since F(d) is continuous, that F(8) is one-to-one; i.e., 

det F(6) * for - t £ $ £ x. This completes the proof of Theorem 2. 

There are other interesting properties of the class L, which are not 

directly relevant to the main results of this paper. For the sake of 

completeness, we mention two generalizations of Theorem 2d, that 
also have well-known scalar counterparts: 

(i) Let JF E L and let a (ff) denote the set of all eigenvalues of 
F(d), -x S 6 g x. Let / be the identity on Iff*. For X any 
complex number, XI — JF has a bounded inverse in L if and 
only if \$ er(ff). 
(ii) Let 0(-) be any function analytic in a neighborhood containing 
o^tf). Then there is an operator g(5) in L which has as its 
transfer matrix the function g[F(8)2. 

A.2 Lemmas 3 and 4 

These lemmas apply to the special case where H(6) = I„ and 
R{6) = R 2 (6). Let x, y, and z be the channel input, output, and noise 
sequences respectively, and let x (Ar) , y (iV) , z (Ar) be the corresponding 
finite sequences. Thus, 



* Note that L„||/(n)|| < x if and only if £»|/„(n) | < °° for 1 ^ i, j ^ s. 
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Letting T N be the whitening matrix denned in (18), we have 
v = T N yW = Ta*<*> + w, 

where £ww' = In... This is precisely the discrete-time version of the 
problem treated in Chapter 8 of Gallager. 1 The results obtained there 
apply here exactly when we use, instead of his Lemma 8.5.2 (the 
Kac-Murdock-Szego theorem), the following discrete-time version: 

Theorem. Let [d] i = 0, ±1, • • • be a sequence of s X s matrices such 
that the Ns X Ns matrix C N = {c*-*}, h 3 - 0, •• •, N - 1, is Hermit- 
ian, and'£k\\ck\\ < c0 - Let v[ m , vtf 1 *, ■■■, vffl be the eigenvalues of C N 
{each counted according to its multiplicity) and let \i(0), A 2 (0), ■ * *, K(9) 
be the eigenvalues of C(0) ■ Et c k e ikt . Let g(-)be any continuous function 
defined on an interval containing the values \\k{Q) : — ir ^ ^ t, k = 1, 
2, ■■-, s}. Then 

1 sN 1 « f* 

lim ± E g(vn 4l gMende. 

ff—» iV jfc=l 4TT k=l J-T 

Furthermore, let D N {x) = \/N {number of eigenvalues VJP ^ x). Then 



lim D N {x) =£-t f 



dO. 

X*(fl)^x 



In the scalar case, 8=1, this theorem represents well-known results, 6 
a simple account of which can also be found in Ref. 7. The validity of 
the theorem for s > 1 follows on verifying that the arguments em- 
ployed in Ref. 7 are valid in general for s ^ 1. 

A.3 Derivation ot (38) 

We show how to obtain the capacity formula given in (38). We will 
do this for the scalar case ; the result for vectors follows similarly. The 
capacity formula justified in Theorem 1 can be stated as follows. 

The channel input and output are sequences x = {z n }-« and 
y = {y n }-*> respectively, related by 

QO 

yn = E hn-kXk + z n , (49) 

fc=-oo 

where h= { /i „}"=_„ is a fixed sequence and z = { z„ } " « is a stationary 
sequence of Gaussian random variables for which Ez n = 0, 

Ez m z m -n = fn, ~ °° < n, m < °° . (50) 
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We write (49) symbolically as 

y = h*x + z, (51) 

where "*" denotes vector convolution. The capacity of this channel is 
given as follows. 

For codes of block length N, say that the code vectors x must 
satisfy 

x n = 0, n$lO,N - 1]. (52a) 

Ir £ 4 ^ S D . (52b) 

jv n =o 

The capacity is then given by 

c - w C </»« (°. *» ^g^ 2 ) ■ ( 53a ) 

Hd(/) = £ Ke^i^f, |/| ^ W, (53b) 

n=— CO 

-£/>(/) = £ f *«*'">*', I /| g TT, (53c) 



where 



and 



are the discrete Fourier transforms of [h n \ and {/"„} respectively, and 
K is the unique solution of 

*» - 2TP r, *> m - (°. * - wM\> ) • (53d) 

A. 3.1 Some facts about band-limited functions 

Before discussing the continuous-time channel, we digress to mention 
several facts about band-limited functions. 

We denote time functions by lower-case letters, e.g., u(t), and the 
corresponding Fourier transform by upper-case letters, e.g., £/(/). 
Thus 

U(f) = f X uiQeWdt, (54a) 

7—00 

and 

u(t) = f K U(f)er**f*df. (54b) 

J — QO 

We shall assume that all functions are square-integrable, and all 
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integrals and infinite sums are limits in the mean. We say that u is 
band-limited to W Hz if U(f) =0, \f\ >W. Let 



sin 2irW 



g»(f) = 



(-w) 



, n = 0, ±1, ±2, •■•, (55) 



be the sampling functions. Note that for k, n = 0, ±1, • • •, 
Qn \2W ) ~ \2W k = n, 



and 



/oo e (inirlW)f \f\<W 

j n {t)e**«dt= Q \f\ >W 



(56a) 
(56b) 



(so that g n is band-limited), and for k, n = 0, ±1, • • • 
r g n (t)g k (t)dt = r G n (f)G* t (f)df 

J -oo / — «0 



= /" e {i*flW){n-k)(lf = 
■/-fT 



0, n 5^ fe, 
2PF, n = k. 



(56c) 



Further, the well-known Sampling Theorem implies that any band- 
limited function, u(t), can be written as 



u(t) = £ u n g n (t), 



where 



Un 2W U \2WJ' 
Further, from (57), and (56c), 

r u\t)dt = 2W £ u\. 

./-oo n =-« 



(57a) 



(57b) 



(57c) 



Let u(t) = £ u n g„(t) and w(fl = £ »„0 n (O be band-limited functions. 
Then their convolution is 

w(0 = f* u(t - \)v(t)dt = £ w n gn(t), (58a) 

J — 00 
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where 

00 

w n = Z u n - m v m , — co < n < co, (58b) 

i.e., w = u*v. 

Finally, let z(t) (— =o< t < *>) be a random process with Ez(t) = 0, 
and covariance 

Ez(t)z(t - t) = v(t) - co < t, T < co . 

Let 

R(f) = r r {t)e iir ^dt 

J —00 

satisfy R(f) = 0, |/| > W, so that r(t) is band-limited. Then, from 

(57), 

and from (56b), 

B <» " ?2W''(2w) G " (/) " 2^^? r (2w) e< " , "" , ■ (59) 

Further, the random process z{t) is a band-limited function so that by 
(57) we can write 

z(0 = Zz n gn(t) = ^2W Z (w) 9n ^' 

Thus, 

. a p 1 „ / m_ \ / m — n \ 1 / n \ 

r n -tLz m z m - n {2W)2 uz y 2W )zy 2W ) (2W) 2l \2W)' 

Thus, the discrete Fourier transform of \f m ) is, using (59), 

fi»(f) = E t^"">" = pF? E »'(^) »«*"■"•' = ^fl(/). (60) 

A.3.2 The continuous-time channel 

The continuous-time channel is denned as follows. The channel in- 
put and output are functions x(t) and y(t), respectively, where 

y (t) = r h(t - \)x(\) dX + z(t), - co < t < co, (61) 

J — 00 

where h(t) is a fixed function and z(t) is a Gaussian random process 
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with covariance as described above. We assume that x(t), h(t), and, 
therefore, y(t) are band-limited to W Hz. Let us expand x, h, z, and y 
into series in g„(t) as in (57). Using (58) we obtain 

00 

y n = E h n - m x m + z m . (62) 

to— oo 

Since knowledge of the sequences of coefficients {x n \, {ijn}, etc. is 
equivalent to knowledge of the time functions x{t), y(t), etc., the 
continuous-time channel is equivalent to the discrete-time channel 
discussed at the beginning of this appendix. It remains to find the 
corresponding parameters. 

Now the code words (in a T-second block-coding interval) are 
taken to be band-limited functions x(t) such that the samples 
x(n/2W) = 0, for (n/2W) < or (n/2W) ^ T, i.e., x(n/2W) = 0, 
n( £[ , N - 1], where N = 2WT. Thus, x n = (l/2W)x(n/2W) = 0, 
n&[0, N - 1]. The condition 

r x 2 (t)dt g ST 

J— to 

is, in the light of (57c), 

- N T x 2 < S • (63) 

The quantity 

H D {f) = t h n e^' w ^ = E h n G n (f) = H(f), 

n=— co n 

and by (60), R D (f) = (l/2W)R(f). Thus, the continuous-time channel 
is equivalent to a discrete-time channel with S D = S/2W, H D (f) 
= H(f), and & D (f) = (l/2W)R(f). Thus, from (53) 

c = wL d/max v ' log2 — soj — ) ' 

where K is the solution to 

S 1 f w js (air 1 fi (/) \ 

Letting K* = 2WK, we have 
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and 



c = X/>™(o,i<*,qg2l'). 



The expression for C is in bits per sampling time. To obtain C in 
bits per second, multiply by 2W. 

A.4 Proof of equation (42) 

Equation (42) follows at once from the theorem below if the func- 
tions \* u i = 1, 2, • • -s, are replaced by a single function / representing 
a concatenation of the functions X*; / will be bounded away from zero 
since the same is true of each X*. 

Theorem: Let /(•) be a measurable function on a finite interval and 
ess inf f{x) = f > 0. For any K > f define A = [x; f(x) ^ K \ 
and let 8 be the measure of A. Define 

log -z J f(x)dx - - [ \ogf(x)dx 

I,(K) = *-±± ^ 

K - \j^ f{x)dx 

Then 

lim I f (K) = 0. 

K-/o 

Proof: Without loss of generality we can take the log to be the natural 
log and can assume f = 1. Let / = 1 + g and K = 1 + k. For each 
n ^ 1 define 

g " = « / g "^ dx - 

For all x £ A, g(x) ^ k and g ^ k. For k < 1, the log may be ex- 
panded in a power series. After some rearrangement of terms, we have 

But Q ^ k and by Jensen's inequality g n ^ g n for all n ^ 1. Then 

£ i ( k2n ~ 2n ) 
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Now for all n ^ 1, 

k" - g n = (k- g) £ fc"-- 1 ^ ^ (k - g)nk»-\ 

t"=0 

and 

or, since if = 1 + ft, 



1 - (#- l) 2 ' 
and the result is proved. 
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